Approximation methods in inductive inference

Deterministic Approximation Methods in Bayesian Inference

In this seminar paper we give an introduction to the field of deterministic approximate inference. We cast the problem of approximating a posterior distribution over hidden variables as a variational minimization problem. In this framework we describe three algorithms: Variational Factorization, Variational Bounds and Expectation Propagation. We analyze the approximations obtained by the three ...

Inductive Inference

Subjectivity in Inductive Inference

This paper examines circumstances under which subjectivity enhances the effectiveness of inductive reasoning. We consider a game in which Fate chooses a data generating process and agents are characterized by inference rules that may be purely objective (or data-based) or may incorporate subjective considerations. The basic intuition is that agents who invoke no subjective considerations are do...

Robust Separations in Inductive Inference

Results in recursion-theoretic inductive inference have been criticized as depending on unrealistic self-referential examples. Barzdin proposed a way of ruling out such examples, and conjectured that one of the earliest results of inductive inference theory would fall if his method were used We refute Barzdin’s conjecture. We propose a new line of research examining robust separations; these ar...

Fixed Parameter Inductive Inference

We investigate a particular inductive inference problem where we are given as input a finite set of ordered triples of the form (wi, bi, ti) such that wi is a bit string, bi is a single bit, and ti is a positive integer written in unary. A machine M is said to match such a triple if M outputs bi on input wi in at most ti steps. We proceed to investigate the fixed parameter problems where we fix...